Alternating linearization for structured regularization problems

OF THE DISSERTATION Alternating linearization for structured regularization problems by Minh Pham Dissertation Director: Andrzej Ruszczyński Xiaodong Lin We adapt the alternating linearization method for proximal decomposition to structured regularization problems. The method is related to two well-known operator splitting methods, the Douglas-Rachford and the Peaceman-Rachford method, but it has descent properties with respect to the objective function. Its convergence mechanism is related to that of bundle methods of nonsmooth optimization. A block coordinate descent method is developed to facilitate fast convergence. We also discuss implementation for large problems, with the use of specialized algorithms and sparse data structures. We present numerical results for several synthetic and real-world examples, including a three-dimensional fused lasso problem, which illustrate the scalability, efficacy, and accuracy of the method. We further extend the alternating linearization framework to the structured regularization problems with non-convex penalties. In this framework, the non-convex part of the objective function is approximated via a linear model. Therefore, in each iteration, the method solves a structured regularization problem. We present several numerical studies with synthetic data and cancer research data.